Versioned File Archiving, Compression, and Distribution

The Xdelta system implements a technique for archiving and compressing collections of many similar le versions. It stores only the diierences between certain versions. I describe and discuss an algorithm for computing le deltas, present measurements, and demonstrate its application to versioned le-archival and eecient le-distribution network protocols. 1 Overview The le delta problem is to compute a small set of instructions for transforming one le into another|one that is expected to be a function of the le's changes, not its content. This technique is well established for versioned le-archival. Though the advantages of using le deltas to transmit changes over a network are clear, specifying and widely deploying such a system eecient enough to justify itself is not as easy as it seems. There are a number of issues to overcome. First, the execution cost of computing and compressing deltas can be prohibitive{a site administrator might rather let everyone on the network suuer through a congested network condition than be CPU-bound while delivering eecient deltas to an uncongested network. Second, eeciently a integrating delta transmission into an application requires work; a server must store and name multiple versions of a le, compute deltas when requested, complicate existing protocols, and be fair|these are not always compatible. Recently there have been advances in delta algorithms, but their use in network delta communication has been slow to follow. This work addresses many of these issues, and nishes with a few new challenges that I am currently working on. I have designed and implemented a decentralized scheme for highly-available, versioned le-archival, with which I have implemented a delta-based distribution and replication protocol built upon a simple, distributed le-archive abstraction. The system, called Xdelta, is implemented as the basis of multiple server, distributed version control in PRCS, the Project Revision Control System 9]. This paper outlines the Xdelta system in three sections: delta algorithms (x2), le-archival (x3), and distribution protocols (x4). Each section will detail related work, discuss previous implementations, outline a part of the system and measure it, and suggest future work. 2 Delta Algorithms The le delta problem is to eeciently compute a delta d that can be stored compactly and used to construct a To le t d from the set of k d From les F d = f d 0 : : : f d k d. The generate operation computes d from F d and t d : …